[Frontend] Skip `stop` in reasoning content #14550

gaocegege · 2025-03-10T11:06:48Z

Skip stop checks in reasoning content. It is based on #14428 . I will rebase after it is merged.

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

# Round 1
messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
response = client.chat.completions.create(
    model=model,
    messages=messages,
    stop="9.8",
)

reasoning_content = response.choices[0].message.reasoning_content
content = response.choices[0].message.content

print("reasoning_content for Round 1:", reasoning_content)
print("content for Round 1:", content)

Output

reasoning_content for Round 1: To determine which number is greater between 9.11 and 9.8, I'll start by aligning their decimal places.

First, I'll write 9.8 as 9.80 to have the same number of decimal places as 9.11.

Next, I'll compare the whole number parts. Both numbers have 9, so they are equal in that aspect.

Then, I'll compare the tenths place. In 9.80, the tenths digit is 8, while in 9.11, it's 1. Since 8 is greater than 1, 9.80 is greater than 9.11.

Therefore, 9.8 is the greater number.

content for Round 1: 

To determine which number is greater between **9.11** and **

github-actions · 2025-03-10T11:06:59Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

ComposerKevin · 2025-03-20T12:10:40Z

Very looking forward to this patch!

gaocegege · 2025-03-21T01:42:37Z

@ComposerKevin This is blocked by several PRs about the reasoning parser. I will try my best.

ComposerKevin · 2025-03-23T11:45:12Z

Could you also add an extra body option like reasoning_stop? It could be useful for manually induced thinking. For example, it will be possible to prompt model like If you're unsure about X, write $PAUSE_THINKING$ something like that.

mergify · 2025-03-27T05:52:38Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @gaocegege.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

gaocegege · 2025-03-28T09:07:06Z

@aarnphm Could you please take a look?

gaocegege · 2025-04-01T04:11:29Z

/cc @mgoin

mergify · 2025-05-13T15:13:49Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @gaocegege.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

github-actions · 2025-08-12T02:25:46Z

This pull request has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this pull request should remain open. Thank you!

mgoin · 2025-08-22T04:09:35Z

Hey @gaocegege, would you want to resolve conflicts and get this merged? Sorry for losing track

gaocegege · 2025-08-22T10:31:33Z

Hey, not 100% sure if it matches the design yet. I'll dig into it and catch up.

Signed-off-by: Ce Gao <cegao@tensorchord.ai>

gaocegege · 2025-09-05T11:06:43Z

PTAL @mgoin

gaocegege · 2025-09-12T06:05:09Z

cc @chaunceyjiang

chaunceyjiang

Thanks~

Signed-off-by: Ce Gao <cegao@tensorchord.ai> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

Signed-off-by: Ce Gao <cegao@tensorchord.ai> Co-authored-by: Chauncey <chaunceyjiang@gmail.com> Signed-off-by: bbartels <benjamin@bartels.dev>

njhill · 2025-09-16T01:02:08Z

vllm/engine/llm_engine.py

                "vllm.llm_engine",
                self.observability_config.otlp_traces_endpoint)

+        # Initialize reasoning parser if reasoning backend is set.


This class is a V0 class and is obsolete.

Oh I see, thanks for the reminder. Should I delete this part of the code? @njhill

@gaocegege @chaunceyjiang
Just to clarify, is this PR intended only for the V0 engine and not compatible with V1? Do you have any plans to add the same feature to V1?

@llsj14 Yes. #24941

Thank you for letting me know!

Signed-off-by: Ce Gao <cegao@tensorchord.ai> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

Signed-off-by: Ce Gao <cegao@tensorchord.ai> Co-authored-by: Chauncey <chaunceyjiang@gmail.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>

Signed-off-by: Ce Gao <cegao@tensorchord.ai> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

Signed-off-by: Ce Gao <cegao@tensorchord.ai> Co-authored-by: Chauncey <chaunceyjiang@gmail.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>

gaocegege mentioned this pull request Mar 10, 2025

[Bug]: If I include the stop parameter, the output of reasoning_content gets truncated, and the output of content will display /think> #14170

Closed

1 task

mergify bot added documentation Improvements or additions to documentation frontend structured-output labels Mar 10, 2025

gaocegege mentioned this pull request Mar 10, 2025

[Bug]: stop_sequences is applied to both reasoning_content and content #14399

Closed

1 task

gaocegege changed the title ~~[Reasoning] Skip stop in reasoning content~~ [Frontend] Skip stop in reasoning content Mar 10, 2025

mergify bot added the tpu Related to Google TPUs label Mar 27, 2025

mergify bot added needs-rebase and removed tpu Related to Google TPUs labels Mar 27, 2025

gaocegege force-pushed the stop branch from 4520afc to e494648 Compare March 28, 2025 09:02

mergify bot removed the needs-rebase label Mar 28, 2025

gaocegege marked this pull request as ready for review March 28, 2025 09:05

gaocegege requested review from alexm-redhat, comaniac, njhill, youkaichao and zhuohan123 as code owners March 28, 2025 09:05

mergify bot added tpu Related to Google TPUs and removed tpu Related to Google TPUs labels Apr 9, 2025

russellb added this to Structured Output Apr 22, 2025

aarnphm approved these changes May 13, 2025

View reviewed changes

mergify bot added the needs-rebase label May 13, 2025

github-actions bot added the stale Over 90 days of inactivity label Aug 12, 2025

github-actions bot added unstale Recieved activity after being labelled stale and removed stale Over 90 days of inactivity labels Aug 23, 2025

[Frontend] Skip stop in reasoning content

0fb102f

Signed-off-by: Ce Gao <cegao@tensorchord.ai>

gaocegege force-pushed the stop branch from e494648 to 0fb102f Compare September 3, 2025 12:40

mergify bot removed the needs-rebase label Sep 3, 2025

gaocegege added 3 commits September 3, 2025 20:55

fix: Add a none check for tokenizer

1e70706

Signed-off-by: Ce Gao <cegao@tensorchord.ai>

fix: Fix the e2e test failure

f72e8a8

Signed-off-by: Ce Gao <cegao@tensorchord.ai>

fix e2e

625bf0a

Signed-off-by: Ce Gao <cegao@tensorchord.ai>

chaunceyjiang approved these changes Sep 12, 2025

View reviewed changes

chaunceyjiang added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 12, 2025

Merge branch 'main' into stop

54040c6

chaunceyjiang enabled auto-merge (squash) September 15, 2025 04:00

chaunceyjiang merged commit f4a948f into vllm-project:main Sep 15, 2025
45 checks passed

github-project-automation bot moved this to Done in Structured Output Sep 15, 2025

dsxsteven pushed a commit to dsxsteven/vllm_splitPR that referenced this pull request Sep 15, 2025

[Frontend] Skip stop in reasoning content (vllm-project#14550)

16f053c

Signed-off-by: Ce Gao <cegao@tensorchord.ai> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

njhill reviewed Sep 16, 2025

View reviewed changes

FeiDaLI pushed a commit to FeiDaLI/vllm that referenced this pull request Sep 25, 2025

[Frontend] Skip stop in reasoning content (vllm-project#14550)

e106e90

Signed-off-by: Ce Gao <cegao@tensorchord.ai> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

choprahetarth pushed a commit to Tandemn-Labs/vllm that referenced this pull request Oct 11, 2025

[Frontend] Skip stop in reasoning content (vllm-project#14550)

f581be9

Signed-off-by: Ce Gao <cegao@tensorchord.ai> Co-authored-by: Chauncey <chaunceyjiang@gmail.com>

Uh oh!

[Frontend] Skip stop in reasoning content #14550

[Frontend] Skip stop in reasoning content #14550

Uh oh!

Conversation

gaocegege commented Mar 10, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 10, 2025

Uh oh!

ComposerKevin commented Mar 20, 2025

Uh oh!

gaocegege commented Mar 21, 2025

Uh oh!

ComposerKevin commented Mar 23, 2025

Uh oh!

mergify bot commented Mar 27, 2025

Uh oh!

gaocegege commented Mar 28, 2025

Uh oh!

gaocegege commented Apr 1, 2025

Uh oh!

mergify bot commented May 13, 2025

Uh oh!

github-actions bot commented Aug 12, 2025

Uh oh!

mgoin commented Aug 22, 2025

Uh oh!

gaocegege commented Aug 22, 2025

Uh oh!

gaocegege commented Sep 5, 2025

Uh oh!

gaocegege commented Sep 12, 2025

Uh oh!

chaunceyjiang left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

njhill Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

chaunceyjiang Sep 16, 2025

Choose a reason for hiding this comment

Uh oh!

llsj14 Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

chaunceyjiang Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

llsj14 Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

[Frontend] Skip `stop` in reasoning content #14550

[Frontend] Skip `stop` in reasoning content #14550

gaocegege commented Mar 10, 2025 •

edited by github-actions bot

Loading